Skip to content

Add benchmarks and optimize html5lib a bit #493

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jun 17, 2020
Merged

Conversation

gsnedders
Copy link
Member

Benchmark master perf
wpt_001 529 us 304 us: 1.74x faster (-43%)
wpt_2d.composite.image.destination-over 1.77 ms 1.09 ms: 1.62x faster (-38%)
wpt_align-content-wrap-002 3.95 ms 3.58 ms: 1.10x faster (-9%)
wpt_background-origin-007-ref 740 us 592 us: 1.25x faster (-20%)
wpt_background_shorthand_css_relative_url 1.37 ms 810 us: 1.69x faster (-41%)
wpt_beforeunload-on-history-back-1 423 us 239 us: 1.77x faster (-44%)
wpt_big5_chars_extra 199 ms 190 ms: 1.05x faster (-4%)
wpt_euckr-encode-form 2.24 ms 2.03 ms: 1.11x faster (-10%)
wpt_fetch.http 1.41 ms 1.26 ms: 1.12x faster (-11%)
wpt_filter-turbulence-invalid-001 3.23 ms 3.01 ms: 1.07x faster (-7%)
wpt_frame-ancestors-self-allow 857 us 502 us: 1.71x faster (-41%)
wpt_grid-auto-fill-rows-001 9.17 ms 8.79 ms: 1.04x faster (-4%)
wpt_grouping-dl 1.03 ms 873 us: 1.17x faster (-15%)
wpt_heavy-styling-005 648 us 505 us: 1.28x faster (-22%)
wpt_htb-ltr-ltr 4.09 ms 3.79 ms: 1.08x faster (-7%)
wpt_idlharness 1.10 ms 656 us: 1.67x faster (-40%)
wpt_image-orientation-from-image-content-images-ref 3.43 ms 2.77 ms: 1.24x faster (-19%)
wpt_li-type-unsupported-ref 816 us 655 us: 1.25x faster (-20%)
wpt_masonry-item-placement-006 4.92 ms 4.54 ms: 1.08x faster (-8%)
wpt_moz-css21-float-page-break-inside-avoid-6 1.25 ms 1.09 ms: 1.15x faster (-13%)
wpt_moz-css21-table-page-break-inside-avoid-2 1.99 ms 1.80 ms: 1.10x faster (-9%)
wpt_position-sticky-table-th-bottom-ref 1.70 ms 1.19 ms: 1.42x faster (-30%)
wpt_pre-float-001 1.96 ms 1.20 ms: 1.64x faster (-39%)
wpt_resize-004 1.05 ms 892 us: 1.18x faster (-15%)
wpt_shape-outside-content-box-002 2.41 ms 2.20 ms: 1.09x faster (-9%)
wpt_test-plan.src 110 ms 104 ms: 1.05x faster (-5%)
wpt_toBlob.png 849 us 517 us: 1.64x faster (-39%)
wpt_will-change-abspos-cb-001 976 us 816 us: 1.20x faster (-16%)
wpt_worker-constructor.https 1.01 ms 857 us: 1.18x faster (-15%)
html_parse 303 ms 290 ms: 1.04x faster (-4%)
Not significant (1): html_serialize

Full revision by revision changes:

(Note that the intermediate versions were done with vastly fewer data points, so some of these may appear significant when they're not)

Benchmark wpt_master wpt_1a3d30e wpt_db57548 wpt_1176bf2 wpt_a723554 wpt_2ed454b wpt_perf
wpt_001 529 us 532 us: 1.01x slower (1%) 527 us: 1.00x faster (-0%) 457 us: 1.16x faster (-14%) 312 us: 1.70x faster (-41%) 309 us: 1.71x faster (-42%) 304 us: 1.74x faster (-43%)
wpt_2d.composite.image.destination-over 1.77 ms 1.74 ms: 1.02x faster (-2%) 1.74 ms: 1.02x faster (-2%) 1.25 ms: 1.42x faster (-29%) 1.13 ms: 1.58x faster (-37%) 1.11 ms: 1.60x faster (-37%) 1.09 ms: 1.62x faster (-38%)
wpt_align-content-wrap-002 3.95 ms 3.86 ms: 1.02x faster (-2%) 3.86 ms: 1.02x faster (-2%) 3.77 ms: 1.05x faster (-5%) 3.66 ms: 1.08x faster (-8%) 3.58 ms: 1.10x faster (-9%) 3.58 ms: 1.10x faster (-9%)
wpt_background-origin-007-ref 740 us 733 us: 1.01x faster (-1%) 732 us: 1.01x faster (-1%) 733 us: 1.01x faster (-1%) 610 us: 1.21x faster (-18%) 601 us: 1.23x faster (-19%) 592 us: 1.25x faster (-20%)
wpt_background_shorthand_css_relative_url 1.37 ms 1.36 ms: 1.00x faster (-0%) 1.35 ms: 1.01x faster (-1%) 958 us: 1.43x faster (-30%) 830 us: 1.64x faster (-39%) 820 us: 1.66x faster (-40%) 810 us: 1.69x faster (-41%)
wpt_beforeunload-on-history-back-1 423 us 427 us: 1.01x slower (1%) 426 us: 1.01x slower (1%) 387 us: 1.09x faster (-8%) 244 us: 1.73x faster (-42%) 243 us: 1.74x faster (-43%) 239 us: 1.77x faster (-44%)
wpt_big5_chars_extra 199 ms 192 ms: 1.04x faster (-3%) 193 ms: 1.03x faster (-3%) 191 ms: 1.04x faster (-4%) 189 ms: 1.05x faster (-5%) 188 ms: 1.06x faster (-5%) 190 ms: 1.05x faster (-4%)
wpt_euckr-encode-form 2.24 ms 2.21 ms: 1.02x faster (-2%) 2.22 ms: 1.01x faster (-1%) 2.23 ms: 1.01x faster (-1%) 2.08 ms: 1.08x faster (-7%) 2.05 ms: 1.09x faster (-9%) 2.03 ms: 1.11x faster (-10%)
wpt_fetch.http 1.41 ms 1.40 ms: 1.01x faster (-1%) 1.40 ms: 1.01x faster (-1%) 1.40 ms: 1.01x faster (-1%) 1.28 ms: 1.10x faster (-9%) 1.27 ms: 1.11x faster (-10%) 1.26 ms: 1.12x faster (-11%)
wpt_filter-turbulence-invalid-001 3.23 ms 3.19 ms: 1.01x faster (-1%) 3.18 ms: 1.02x faster (-2%) 3.18 ms: 1.02x faster (-2%) 3.05 ms: 1.06x faster (-5%) 3.03 ms: 1.07x faster (-6%) 3.01 ms: 1.07x faster (-7%)
wpt_frame-ancestors-self-allow 857 us 854 us: 1.00x faster (-0%) 849 us: 1.01x faster (-1%) 651 us: 1.32x faster (-24%) 516 us: 1.66x faster (-40%) 510 us: 1.68x faster (-41%) 502 us: 1.71x faster (-41%)
wpt_grid-auto-fill-rows-001 9.17 ms 9.10 ms: 1.01x faster (-1%) 9.09 ms: 1.01x faster (-1%) 9.05 ms: 1.01x faster (-1%) 8.91 ms: 1.03x faster (-3%) 8.81 ms: 1.04x faster (-4%) 8.79 ms: 1.04x faster (-4%)
wpt_grouping-dl 1.03 ms 1.02 ms: 1.01x faster (-1%) 1.02 ms: 1.00x faster (-0%) 1.02 ms: 1.01x faster (-1%) 898 us: 1.14x faster (-12%) 887 us: 1.16x faster (-13%) 873 us: 1.17x faster (-15%)
wpt_heavy-styling-005 648 us 648 us: 1.00x slower (0%) 648 us: 1.00x slower (0%) 649 us: 1.00x slower (0%) 517 us: 1.25x faster (-20%) 510 us: 1.27x faster (-21%) 505 us: 1.28x faster (-22%)
wpt_htb-ltr-ltr 4.09 ms 4.07 ms: 1.00x faster (-0%) 4.06 ms: 1.01x faster (-1%) 4.05 ms: 1.01x faster (-1%) 3.86 ms: 1.06x faster (-6%) 3.82 ms: 1.07x faster (-7%) 3.79 ms: 1.08x faster (-7%)
wpt_idlharness 1.10 ms 1.09 ms: 1.01x faster (-1%) 1.08 ms: 1.01x faster (-1%) 811 us: 1.35x faster (-26%) 670 us: 1.64x faster (-39%) 661 us: 1.66x faster (-40%) 656 us: 1.67x faster (-40%)
wpt_image-orientation-from-image-content-images-ref 3.43 ms 3.36 ms: 1.02x faster (-2%) 3.36 ms: 1.02x faster (-2%) 2.99 ms: 1.15x faster (-13%) 2.88 ms: 1.19x faster (-16%) 2.83 ms: 1.21x faster (-17%) 2.77 ms: 1.24x faster (-19%)
wpt_li-type-unsupported-ref 816 us 811 us: 1.01x faster (-1%) 820 us: 1.00x slower (0%) 816 us: 1.00x faster (-0%) 678 us: 1.20x faster (-17%) 663 us: 1.23x faster (-19%) 655 us: 1.25x faster (-20%)
wpt_masonry-item-placement-006 4.92 ms 4.87 ms: 1.01x faster (-1%) 4.86 ms: 1.01x faster (-1%) 4.90 ms: 1.00x faster (-0%) 4.75 ms: 1.03x faster (-3%) 4.66 ms: 1.06x faster (-5%) 4.54 ms: 1.08x faster (-8%)
wpt_moz-css21-float-page-break-inside-avoid-6 1.25 ms 1.25 ms: 1.00x faster (-0%) 1.24 ms: 1.01x faster (-1%) 1.24 ms: 1.01x faster (-1%) 1.11 ms: 1.13x faster (-11%) 1.10 ms: 1.13x faster (-12%) 1.09 ms: 1.15x faster (-13%)
wpt_moz-css21-table-page-break-inside-avoid-2 1.99 ms 1.96 ms: 1.01x faster (-1%) 1.94 ms: 1.02x faster (-2%) 1.94 ms: 1.03x faster (-2%) 1.82 ms: 1.09x faster (-9%) 1.80 ms: 1.10x faster (-9%) 1.80 ms: 1.10x faster (-9%)
wpt_position-sticky-table-th-bottom-ref 1.70 ms 1.68 ms: 1.01x faster (-1%) 1.66 ms: 1.02x faster (-2%) 1.36 ms: 1.25x faster (-20%) 1.23 ms: 1.38x faster (-28%) 1.21 ms: 1.40x faster (-29%) 1.19 ms: 1.42x faster (-30%)
wpt_pre-float-001 1.96 ms 1.94 ms: 1.01x faster (-1%) 1.91 ms: 1.03x faster (-3%) 1.37 ms: 1.43x faster (-30%) 1.25 ms: 1.57x faster (-36%) 1.22 ms: 1.61x faster (-38%) 1.20 ms: 1.64x faster (-39%)
wpt_resize-004 1.05 ms 1.04 ms: 1.01x faster (-1%) 1.05 ms: 1.01x faster (-1%) 1.05 ms: 1.01x faster (-1%) 919 us: 1.15x faster (-13%) 904 us: 1.17x faster (-14%) 892 us: 1.18x faster (-15%)
wpt_shape-outside-content-box-002 2.41 ms 2.39 ms: 1.01x faster (-1%) 2.39 ms: 1.01x faster (-1%) 2.39 ms: 1.01x faster (-1%) 2.26 ms: 1.06x faster (-6%) 2.23 ms: 1.08x faster (-7%) 2.20 ms: 1.09x faster (-9%)
wpt_test-plan.src 110 ms 108 ms: 1.01x faster (-1%) 109 ms: 1.01x faster (-1%) 109 ms: 1.01x faster (-1%) 108 ms: 1.01x faster (-1%) 106 ms: 1.03x faster (-3%) 104 ms: 1.05x faster (-5%)
wpt_toBlob.png 849 us 844 us: 1.01x faster (-1%) 839 us: 1.01x faster (-1%) 662 us: 1.28x faster (-22%) 529 us: 1.60x faster (-38%) 523 us: 1.62x faster (-38%) 517 us: 1.64x faster (-39%)
wpt_will-change-abspos-cb-001 976 us 966 us: 1.01x faster (-1%) 965 us: 1.01x faster (-1%) 967 us: 1.01x faster (-1%) 839 us: 1.16x faster (-14%) 825 us: 1.18x faster (-15%) 816 us: 1.20x faster (-16%)
wpt_worker-constructor.https 1.01 ms 1.00 ms: 1.00x faster (-0%) 1.01 ms: 1.00x faster (-0%) 1.00 ms: 1.01x faster (-1%) 872 us: 1.15x faster (-13%) 865 us: 1.16x faster (-14%) 857 us: 1.18x faster (-15%)
html_parse 303 ms 300 ms: 1.01x faster (-1%) 299 ms: 1.01x faster (-1%) 299 ms: 1.02x faster (-1%) 298 ms: 1.02x faster (-2%) 298 ms: 1.02x faster (-2%) 290 ms: 1.04x faster (-4%)
html_serialize 105 ms 105 ms: 1.00x slower (0%) 105 ms: 1.00x faster (-0%) 106 ms: 1.01x slower (1%) 105 ms: 1.00x slower (0%) 105 ms: 1.00x slower (0%) not significant

Essentially, the majority of the gains come from avoiding the entire meta pre-parse when the pre-parse range doesn't contain <meta and from only initializing the method dispatcher once (and merely binding it to the phase instance when that is created; this is especially significant when parsing small inputs as it is constant overhead).

@gsnedders gsnedders requested a review from jgraham May 20, 2020 15:11
@gsnedders gsnedders self-assigned this May 20, 2020
@codecov-commenter
Copy link

codecov-commenter commented May 20, 2020

Codecov Report

Merging #493 into master will increase coverage by 0.14%.
The diff coverage is 97.55%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #493      +/-   ##
==========================================
+ Coverage   90.97%   91.11%   +0.14%     
==========================================
  Files          49       50       +1     
  Lines        6936     7036     +100     
  Branches     1322     1340      +18     
==========================================
+ Hits         6310     6411     +101     
- Misses        468      471       +3     
+ Partials      158      154       -4     
Impacted Files Coverage Δ
html5lib/tests/tokenizer.py 82.09% <ø> (ø)
html5lib/_utils.py 84.88% <77.27%> (-2.62%) ⬇️
html5lib/treebuilders/etree_lxml.py 82.08% <93.93%> (+1.93%) ⬆️
html5lib/_inputstream.py 91.77% <100.00%> (ø)
html5lib/_tokenizer.py 99.59% <100.00%> (+<0.01%) ⬆️
html5lib/html5parser.py 98.43% <100.00%> (-0.02%) ⬇️
html5lib/tests/test_parser2.py 100.00% <100.00%> (ø)
html5lib/tests/test_tokenizer2.py 100.00% <100.00%> (ø)
html5lib/tests/test_treewalkers.py 98.13% <100.00%> (+0.94%) ⬆️
html5lib/treebuilders/etree.py 76.51% <100.00%> (+1.00%) ⬆️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d49afd3...86530b9. Read the comment docs.

@gsnedders
Copy link
Member Author

@jgraham I think this can land now?

@gsnedders gsnedders merged commit b2b08cc into html5lib:master Jun 17, 2020
@gsnedders gsnedders deleted the perf branch June 17, 2020 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants